Hypothesis Selection in Grammar Acquisition

This pat)er t)resents some techniques for selecting linguistically adequate hypol;heses of new grammat.foal knowledge to be used as resources of gralmnatical knowledge acquisition. In our framework of liuguistic knowledge acquisition, a rulebased hypothesis generator is inw)ked in case of parsing failures and all the possible hypotheses of Llew graliLlnar rules or lexical entries are generated ffOln partial parsing results. Although each hypothesis could recover the (lef>cl, s of the existing grammar, the greater part of hypotheses are linguistically unnatural. The techniques we propose here prevent such unnatural hypotheses Dont being gene,'ated without discarding plausible ones and make the following corpus-based acquisition l)rocess In(ire ellieien(, and more reliable. 1 I n t r o d u c t i o n Reusability of existing linguistic knowledge is the most import,~mt requirement for the rapid development of pra.ctical nal, ural ]augtlage l)rocessing systems. In order to realize, automatic customizatiou of existing linguistic knowledge to each applicat;ion domain, we proposed a new approach of linguistic knowledge acquisition, which is a combination of symbolic and statistical approaches [Kiyono and Tsujii, 1993]. The fi:amework of our al)proach is shown in Figure 1. '1'111". acquisilion flow starl;s with executing the l)arse of each sentence in a corpus. If parsing lhiled, |,he 'tiypoi;hesis Generator ' produces the hyl)otheses of additional gramnu~tical knowh;dge, each of which could recover t;lle incompleteness of the existing g rammar After iterating t;his hypothesis generation process for all the senten('es in the corpus, the hypotheses are passed to the statistical analysis procc.ss and finally plausible hypotheses are chosen as new knowh'.dge by observing statistical properties of tile hypotheses. Unlike robusl; parsing [Mellish, 1989; Goeser, 1992; l)ouglas and I)ale, 1 !)92] or nou-statisl.ical alll)roach for grallunar a(:lluisil;ioll , our al/proach does Ilol; require a mechanism to detect tile cause of the parsing fail-ure in the sentencial analysis phase and therefore the 'Ilypothesis (~eneral;or' may output ;111 I,he possible hypotheses, l[owever, the greater part 1)t' hypotheses generated by a simple deductive mechanism are unnatural revisions of the e.xisting grammar. For example, even ~ rule which derives a tot) node category ,9 direcl,ly from the input string of words might be hypol,hesize(I. *a.lso a staff member ol7 Mattsus|dt~t Electric lndustri~d Co.,Ltd., Shina.gawa, Tokyo, .IAPAN. Q__._ C°rlnts ) i